Interactive safety system for vehicles

ABSTRACT

An interactive vehicle safety system having capabilities to improve peripheral vision, provide warning, and improve reaction time for operators of vehicles. For example, the interactive vehicle safety system may have capabilities for portraying objects, which are being blocked by any of the structural pillars and/or mirrors of a vehicle (such as a truck, van, train, etc.). The interactive vehicle safety system disclosed may comprise one or more image capturing devices (such as camera, sensor, laser), distance and object sensors (such as ultrasonic sensor, LIDAR radar sensor, photoelectric sensor, and infrared sensor), a real-time image processing of an object, and one or more display systems (such as LCD or LED displays). The interactive vehicle safety system may give a seamless 360-degree front panoramic view to a driver.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent No.62/932,188, filed Nov. 7, 2019, entitled Interactive Safety System forVehicles, which is incorporated herein by reference in its entirety andmade a part hereof.

FIELD OF THE INVENTION

The present invention relates vehicles, and more particularly, aninteractive vehicle safety system having capabilities to improveperipheral vision, provide warning, and improve reaction time foroperators of vehicles.

BACKGROUND

Currently, there are a number of vehicle collisions and accidentsrelated to reduced peripheral vision. A system is needed to improvesafety for vehicles by improving peripheral vision and situationalawareness to reduce vehicle collisions and accidents. A system is alsoneeded with predictive capabilities for vehicles, people, and otherobjects that will help the driver to make initial decisions and have themachine or system take over the decision-making and actions when thedriver is making a mistake (i.e. an imminent accident or collision).This is especially needed with bigger vehicles, for example trucks,trains, etc.

SUMMARY

The present invention generally provides an interactive vehicle safetysystem having capabilities to improve peripheral vision, providewarning, and improve reaction time for operators of vehicles, there area number of vehicle collisions and accidents related to reducedperipheral vision.

According to one embodiment, an interactive safety system for vehiclesmay comprise: one or more image capturing devices located on the vehicleconfigured to transmit one or more images from the vehicle to aprocessor; one or more object detecting sensors located on the vehiclethat detect location, speed, and direction data of one or more objectsexternal to the vehicle; and a display system connected to theprocessor. The one or more object detecting sensors may be configured totransmit the location, speed, and direction data from the one or moreobjects to the processor. The display system may comprise a pillardisplay located on an A-pillar of the vehicle configured to portray animage blocked by an obstruction of the A-pillar from the one or moreimage capturing devices. The processor and a memory unit storingcomputer-executable instructions, which when executed by the processor,may cause the processor to: receive images from the one or more imagecapturing devices; receive the location, speed, and direction data fromthe one or more object detecting sensors; process in real-time theimages and the location, speed, and direction data from the one or moreobjects external to the vehicle; track in real-time the one or moreobjects external to the vehicle using the images and the location,speed, and direction data; predict a future location and a route of theone or more objects external to the vehicle using a predictive algorithmand a trajectory analysis of the one or more objects external to thevehicle; determine in real-time a danger object from the one or moreobjects external to the vehicle, wherein the danger object is apotential collision based on the predictive algorithm and trajectoryanalysis; and display the image blocked by the obstruction of theA-pillar on the pillar display.

Additionally, the processor and the memory unit storingcomputer-executable instructions may cause the processor to furtherprovide a visual warning to the danger object, wherein the visualwarning is provided external to the vehicle. The processor and thememory unit storing computer-executable instructions may cause theprocessor to further provide an audible warning to the danger object,wherein the audible warning is provided external to the vehicle, whereinthe audible warning is a 3D sound directed to the danger object.Additionally, the one or more object detecting sensors may be ultrasonicsensors, LIDAR radar sensors, or photoelectric sensors. The processorand the memory unit storing computer-executable instructions may causethe processor to further light a path of the vehicle with high-intensitylights based on the predictive algorithm and trajectory analysis,wherein the path is externally outside the vehicle on a pavement of astreet. Additionally, the display system may further include a heads-updisplay on a windshield of the vehicle to display the one or moreobjects with the location, speed, and direction data and the dangerobject with the future location and the route on a heads-up display. Thedisplay system may further include a dashboard display on a dashboard ofthe vehicle to display the one or more objects with the location, speed,and direction data and the danger object with the future location andthe route on a heads-up display.

According to another embodiment, an interactive safety system forvehicles may comprise: one or more image capturing devices located onthe vehicle configured to transmit one or more images from the vehicleto a processor; one or more object detecting sensors located on thevehicle that detect location, speed, and direction data of one or moreobjects external to the vehicle; and a display system connected to theprocessor. The display system may include a pillar display located on anA-pillar of the vehicle configured to portray an image blocked by anobstruction of the A-pillar from the one or more image capturingdevices; and a rear-view display located in the vehicle configured toportray an image that would be seen in a rear-view mirror from the oneor more image capturing devices. Further, the one or more objectdetecting sensors may be configured to transmit the location, speed, anddirection data from the one or more objects to the processor. Theprocessor and a memory unit storing computer-executable instructions,which when executed by the processor, may cause the processor to:receive images from the one or more image capturing devices; receive thelocation, speed, and direction data from the one or more objectdetecting sensors; process in real-time the images and the location,speed, and direction data from the one or more objects external to thevehicle; track in real-time the one or more objects external to thevehicle using the images and the location, speed, and direction data;predict a future location and a route of the one or more objectsexternal to the vehicle using a predictive algorithm and a trajectoryanalysis of the one or more objects external to the vehicle; determinein real-time a danger object from the one or more objects external tothe vehicle, wherein the danger object is a potential collision based onthe predictive algorithm and trajectory analysis; display the imageblocked by the obstruction of the A-pillar on the pillar display; anddisplay the image that would be seen in the rear-view mirror on therear-view display.

In yet a further embodiment, an interactive safety system for vehiclesmay comprise: one or more image capturing devices located on the vehicleconfigured to transmit one or more images from the vehicle to aprocessor; one or more object detecting sensors located on the vehiclethat detect location, speed, and direction data of one or more objectsexternal to the vehicle; one or more telematics devices configured totransmit vehicle telematics data from the vehicle to the processor; anda display system connected to the processor. The display system mayinclude: a pillar display located on an A-pillar of the vehicleconfigured to portray an image blocked by an obstruction of the A-pillarfrom the one or more image capturing devices; a rear-view displaylocated in the vehicle configured to portray an image that would be seenin a rear-view mirror from the one or more image capturing devices; andheads-up display on a windshield of the vehicle to display the one ormore objects with the location, speed, and direction data. Additionally,the one or more object detecting sensors configured to transmit thelocation, speed, and direction data from the one or more objects to theprocessor. The processor and a memory unit storing computer-executableinstructions, which when executed by the processor, may cause theprocessor to: receive images from the one or more image capturingdevices; receive the location, speed, and direction data from the one ormore object detecting sensors; receive the telematics data from thetelematics device; process in real-time the images, the telematics data,and the location, speed, and direction data from the one or more objectsexternal to the vehicle; track in real-time the one or more objectsexternal to the vehicle using the images and the location, speed, anddirection data; predict a future location and a route of the one or moreobjects external to the vehicle using a predictive algorithm and atrajectory analysis of the one or more objects external to the vehicle;determine in real-time a danger object from the one or more objectsexternal to the vehicle, wherein the danger object is a potentialcollision based on the predictive algorithm and trajectory analysis;highlight the danger object on the display system; display the imageblocked by the obstruction of the A-pillar on the pillar display;display the image that would be seen in the rear-view mirror on therear-view display; and display the one or more objects with thelocation, speed, and direction data and the danger object with thefuture location and the route on the heads-up display. Further, thedanger object may be highlighted yellow if the danger object ispotentially in a path of an accident or a collision and the dangerobject may be highlighted red if the danger object is imminently in apath of an accident or a collision with the vehicle. The highlighting ofthe danger object may include one or more of the following: blinkingcolors on the danger object, circles around the danger object, orblinking circles around the danger object.

In an additional embodiment, an interactive safety system for vehiclesmay comprise: one or more image capturing devices located on the vehicleconfigured to transmit one or more images from the vehicle to aprocessor; one or more object detecting sensors located on the vehiclethat detect location, speed, and direction data of one or more objectsexternal to the vehicle; and a display system connected to theprocessor. The display system may include a pillar display located on anA-pillar of the vehicle configured to portray an image blocked by anobstruction of the A-pillar from the one or more image capturingdevices. The one or more object detecting sensors may be configured totransmit the location, speed, and direction data from the one or moreobjects to the processor. The processor and a memory unit storingcomputer-executable instructions, which when executed by the processor,may cause the processor to: receive images from the one or more imagecapturing devices; receive the location, speed, and direction data fromthe one or more object detecting sensors; process in real-time theimages and the location, speed, and direction data from the one or moreobjects external to the vehicle; track in real-time the one or moreobjects external to the vehicle using the images and the location,speed, and direction data; predict a future location and a route of theone or more objects external to the vehicle using a predictive algorithmand a trajectory analysis of the one or more objects external to thevehicle; determine in real-time a danger object from the one or moreobjects external to the vehicle, wherein the danger object is apotential collision based on the predictive algorithm and trajectoryanalysis; display the image blocked by the obstruction of the A-pillaron the pillar display; and provide an audible warning to an operatorcoming from a location of the danger object, wherein the audible warningis an audio spotlight from an aimed audio device connected to theprocessor that casts a sound to a surface redirecting the sound to comefrom the location of the danger object to the vehicle.

Other features and advantages of the invention will be apparent from thefollowing specification taken in conjunction with the followingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

To understand the present invention, it will now be described by way ofexample, with reference to the accompanying drawings in which:

FIG. 1 illustrates a prior art version of a vehicle with a side rearviewmirror and an A-pillar;

FIG. 2 illustrates a vehicle with an interactive vehicle safety systemwith side rearview mirror elimination in accordance with an embodimentof the present invention;

FIG. 3 illustrates a vehicle with an interactive vehicle safety systemhaving pillar obstruction elimination in accordance with an embodimentof the present invention;

FIG. 4 illustrates a vehicle with an interactive vehicle safety systemhaving both the side rearview mirror elimination and pillar obstructionelimination in accordance with an embodiment of the present invention;

FIGS. 5A-5D illustrate top views of various vehicles with an interactivevehicle safety system in accordance with an embodiment of the presentinvention;

FIG. 6 illustrates an illustrative system depiction of an interactivevehicle safety system in accordance with an embodiment of the presentinvention;

FIGS. 7A1 and 7A2 illustrate an interactive vehicle safety system withtrajectory and dead-reckoning analysis showing a second vehicle turningleft in front of a main vehicle in accordance with an embodiment of thepresent invention;

FIGS. 7B1 and 7B2 illustrate an interactive vehicle safety system withtrajectory and dead-reckoning analysis showing a main vehicle turningleft in front of a second vehicle in accordance with an embodiment ofthe present invention;

FIG. 7C illustrates an interactive vehicle safety system with trajectoryand dead-reckoning analysis showing a pedestrian walking in a crosswalkin front of a main vehicle in accordance with an embodiment of thepresent invention;

FIGS. 7D1 and 7D2 illustrate an interactive vehicle safety system withtrajectory and dead-reckoning analysis showing a main vehicle turningleft in front of a pedestrian in a crosswalk in accordance with anembodiment of the present invention;

FIG. 8A illustrates the interactive vehicle safety system from FIGS. 7A1and 7A2 with a heads-up-display (HUD) in accordance with an embodimentof the present invention;

FIG. 8B illustrates the interactive vehicle safety system from FIGS. 7B1and 7B2 with a heads-up-display (HUD) in accordance with an embodimentof the present invention;

FIG. 8C illustrates the interactive vehicle safety system from FIG. 7Cwith a heads-up-display (HUD) in accordance with an embodiment of thepresent invention;

FIG. 8D illustrates the interactive vehicle safety system from FIGS. 7D1and 7D2 with a heads-up-display (HUD) in accordance with an embodimentof the present invention;

FIG. 9 illustrates an interactive vehicle safety system with a displayin the dashboard in accordance with an embodiment of the presentinvention;

FIG. 10 illustrates an interactive vehicle safety system using hologramtechnology in accordance with an embodiment of the present invention;

FIG. 11 illustrates an interactive vehicle safety system using audiospotlight technology in accordance with an embodiment of the presentinvention; and

FIG. 12 illustrates an interactive vehicle safety system using anexternal highlighting technology in accordance with an embodiment of thepresent invention.

The reader is advised that the attached drawings are not necessarilydrawn to scale.

DETAILED DESCRIPTION OF EMBODIMENTS

In the following description of various examples of the invention,reference is made to the accompanying drawings, which form a parthereof, and in which are shown by way of illustration various examplestructures, systems, and steps in which aspects of the invention may bepracticed. It is to be understood that other specific arrangements ofparts, structures, example devices, systems, and steps may be utilizedand structural and functional modifications may be made withoutdeparting from the scope of the present invention. Also, while the terms“top,” “bottom,” “front,” “back,” “side,” and the like may be used inthis specification to describe various example features and elements ofthe invention, these terms are used herein as a matter of convenience,e.g., based on the example orientations shown in the figures. Nothing inthis specification should be construed as requiring a specificthree-dimensional orientation of structures in order to fall within thescope of this invention.

The present invention relates to vehicles, and more particularly, aninteractive vehicle safety system having capabilities to improveperipheral vision, provide warning, and improve reaction time foroperators of vehicles. For example, the interactive vehicle safetysystem may have capabilities for portraying objects, which are beingblocked by any of the structural pillars and/or mirrors of a vehicle(such as a truck, van, train, etc.). The interactive vehicle safetysystem disclosed may comprise one or more image capturing devices (suchas camera, sensor, laser), distance and object sensors (such asultrasonic sensor, LIDAR radar sensor, photoelectric sensor, andinfrared sensor), a real-time image processing of an object, and one ormore display systems (such as LCD or LED displays). The interactivevehicle safety system may give a seamless 360-degree front panoramicview to a driver.

The invention relates to an interactive vehicle safety system thatincludes real-time image processing for a vehicle with clear metaltechnology. Clear metal technology refers to the capability forportraying objects, which are being blocked by any of the structuralpillars and/or mirrors of a vehicle. The clear metal technology may useone or multiple cameras located on the other side of a structural pillaror obstruction blocking the vision of the operator to portray an imageblocked by the structural pillar/obstruction on the structuralpillar/obstruction of the vehicle to make the structuralpillar/obstruction functionally disappear. The interactive vehiclesafety system and clear metal technology may be used with vehicles suchas automobiles, trucks, trains, bicycles, motorcycles, trains,forklifts, etc.

The interactive vehicle safety system may include one or more featuressuch as: changing the image on the screen as the vehicle driver's headmoves (using one camera or multiple cameras in “stereo” tracking thedriver's head position); detect and track in real-time object movementoutside the vehicle and highlight stationary or moving objects aroundthe vehicle that may be “danger” by using trajectory analysis and deadreckoning of the vehicle (speed, direction, turn radius, etc.) and/orusing trajectory analysis and dead reckoning of the moving objects(speed, direction, etc.); and a dead reckoning heads-up display toinclude object detecting sensors (for example, ultrasonic sensor, LIDARradar sensor, photoelectric sensor, and infrared sensor) to the frontbumper, rear bumper, or other locations around the vehicle that providesa dead reckoning time fully across the windshield. The interactivevehicle safety system may utilize sensors on the vehicle to senseobjects approaching to turn-on the clear metal technology and turn offthe clear metal technology if nothing is sensed by the sensors.

FIG. 1 illustrates a prior art version of the inside of a vehicle 10. Asillustrated in the prior art, the vehicle 10 includes a large rear-viewmirror 12 and a structural A-pillar 14 on the vehicle 10. FIG. 1demonstrates the need for a system to improve the peripheral vision ofthe driver from the blocked view of the rear-view mirror 12 and/or oneor more of the structural pillars 14 of the vehicle 10.

FIG. 2 illustrates a vehicle 10 with an interactive vehicle safetysystem 100 with a side rearview mirror elimination system 200. The sidemirror elimination system 200 of the interactive vehicle safety system100 of FIG. 2 may include one or more image capturing devices 210 (suchas a camera, sensor, or laser) to take the place of the rearview mirror12, real-time image processing, and one or more display systems 220(such as an LCD or LED display) to portray an image 222 that would beseen on the rear-view mirror 12. In one embodiment the display system220 may be a retina scanner display. In another example, the display 220may be flat, flexible, bendable, or curved without departing from thisinvention. As illustrated in FIG. 2, the display system 220 may belocated on the A-pillar 14. The one or more displays 220 may be locatedin other locations without departing from this invention, such aslocated on another structural pillar, the dashboard 16, or as part of aheads-up-display or other locations.

FIG. 3 illustrates a vehicle 10 with an interactive vehicle safetysystem 100 having a pillar obstruction elimination system 250 andcapabilities for portraying objects that are being blocked by anA-pillar 14 (or other structural pillars) of the vehicle 10. The pillarobstruction elimination system 250 of the interactive vehicle safetysystem 100 of FIG. 3 may include one or more image capturing devices(such as a camera, sensor, or laser) to portray an image 262 blocked bythe obstruction of the A-pillar 14 on the vehicle 10 to make thestructural pillar/obstruction 14 functionally disappear. The interactivevehicle safety system 100 may include real-time image processing and oneor more display systems 260 (such as an LCD or LED display) to portraythe image 262 that blocked by the A-pillar 14. In one embodiment, thedisplay system 260 may be a retina scanner display. In another example,the display 260 may be flat, flexible, bendable, or curved withoutdeparting from this invention. As illustrated in FIG. 3, the displaysystem 260 may be located on the A-pillar 14. The display 260 may belocated in other locations without departing from this invention, suchas located on another structural pillar, the dashboard 16, or as part ofa heads-up-display or other locations.

FIG. 4 illustrates a vehicle 10 with an interactive vehicle safetysystem 100 having both a side rearview mirror elimination system 200 anda pillar obstruction elimination system 260 for portraying objects thatare being blocked by an A-pillar 14 or other structural pillar of thevehicle 10. The interactive vehicle safety system 100 of FIG. 4 mayinclude one or more image capturing devices 210 (such as a camera,sensor, or laser) to take the place of the rearview mirror.Additionally, the interactive vehicle safety system 100 of FIG. 4 mayinclude one or more image capturing devices (such as a camera, sensor,or laser) to portray an image 262 blocked by the obstruction of theA-pillar 14 on the vehicle 10 to make the structural pillar/obstruction14 functionally disappear. The interactive vehicle safety system 100 mayinclude real-time image processing and a display system with one or moredisplays 220, 260 (such as an LCD or LED display) to portray the image222 that would be seen on the rear-view mirror 12 and to portray theimage 262 that blocked by the A-pillar 14 or other structural pillar. Inone embodiment the displays 220, 260 may be a retina scanner display. Inanother example, the displays 220, 260 may be flat, flexible, bendable,or curved without departing from this invention. As illustrated in FIG.4, the display system 220, 260 may be located on the A-pillar 14. Thedisplay 220, 260 may be located in other locations without departingfrom this invention, such as located on another structural pillar, thedashboard 16, or as part of a heads-up-display or other locations.

In general, the interactive vehicle safety system 100 may help anoperator detect a collision and provide further information to eliminateor reduce the risks in many different areas, such as: providing visionto the operator, predicting when or how long of a reaction time beforean accident occurs, providing audible warnings to the public andoperator, providing visual warnings to the public and operator,determining and providing a suggested escape route or action, anddetermining and providing any automatic vehicle operations in responseto the imminent collision (i.e. automatic braking and/or steering).

FIGS. 5A-5D illustrate top views of various vehicles with an interactivevehicle safety system 100 and various input systems. FIG. 5A illustratesa top view of an automobile 10A with the interactive vehicle safetysystem 100. FIG. 5B illustrates a top view of a pick-up truck 10B withthe interactive vehicle safety system 100. FIG. 5C illustrates a topview of a delivery truck 10C with the interactive vehicle safety system100. FIG. 5D illustrates a top view of a semi-truck 10D with theinteractive vehicle safety system 100. The interactive vehicle safetysystem 100 may be utilized with any vehicle without departing from thisinvention. The interactive vehicle safety system 100 may provide anoperator with vision and situational awareness to what is happeningexternal to the vehicle 10 by providing peripheral visual awareness. Asillustrated in FIGS. 5A-5D, the interactive vehicle safety system 100and vehicles 10A, 10B, 10C, 10D may include one or more of thefollowing: cameras 124, object detecting sensors 120, ultrasonic sensors122, and vehicle telematic sensors 130, etc. Other inputs may beincluded with the interactive vehicle safety system 100 and vehicles10A, 10B, 10C, 10D as described and illustrated in FIG. 6. The one ormore cameras 124, object detecting sensors 120, etc. and other inputdata devices as listed and described with FIG. 6 may be located atvarious locations throughout the vehicle 10, such as on the frontbumper, rear bumper or rear area, side of the vehicle 10, on top of thevehicle 10, under the vehicle, within the inside of the vehicle 10, orany other locations that can provide meaningful inputs to theinteractive vehicle safety system 100.

FIG. 6 illustrates an illustrative system depiction of the interactivevehicle safety system 100. The interactive vehicle safety system 100 mayinclude a processor 105 that includes a processing unit and a systemmemory to store and execute software instructions. The interactivevehicle safety system 100 may provide various data inputs 110 andprovide various outputs 120 to predict a potential incident, determinedanger, and inform the vehicle 10 and operator to slow down, turn, orstop. As illustrated in FIG. 6, the various inputs 110 to the processor105 and the interactive vehicle safety system 100 may include one ormore of the following: depth camera 112, lasers 114, accelerometerdevice 116, aimed audio device 118, object detecting sensors 120,ultrasonic sensor 122, cameras 124, LIDAR radar sensor 126,photoelectric sensor 128, telematics device 130, infrared sensor 132,internet of things (IoT) 132, or GPS device 134. The interactive vehiclesafety system 100 may include various image capturing devices, such ascameras 124 to capture what is happening in the real world external tothe vehicle 10 and bringing in and highlighting what may happen in thenear future. Additionally, the interactive vehicle safety system 100 mayinclude LIDAR radar 126 and other object detecting sensors 120 (such asultrasonic sensor 122, photoelectric sensor 128, and infrared sensor132). Other sensors and inputs 110 may be utilized for the interactivevehicle safety system 100, such as GPS 136, vehicle telematic sensors140, Internet of Things (IoT) 132 information. Vehicle telematicssensors 140 may monitor the vehicle 10 by using GPS 136 and onboarddiagnostics to record movements on a computerized map, such as with aGPS receiver, and engine interface, an input/output interface (expanderport) in the vehicle 10, a SIM card, or an accelerometer 116. Theinteractive vehicle safety system 100 will gather data from these inputsand the real world to help the operator see obstacles and provide theoperator information to be able to react to obstacles.

The processor 105 may include a processing unit and a system memory tostore and execute software instructions. The various inputs 110 andoutputs 150 may be connected to the processor 105. Additionally, theprocessor 105 may be in communication with and connected to othervarious computer systems. The processor 105 of the interactive vehiclesafety system 100 may have various outputs 150 after processing thevarious inputs 110. The outputs 150 of the interactive vehicle safetysystem 100 may include one or more of the following: audio spotlight152, hologram display 154, heads-up display 156, LED display 158,display system 160, LCD display 162, dashboard display 164, hapticwarning 166, audible warnings 168, image analysis 170, augmented realitydisplay 172, external visual warnings 174, or accident prediction 176.

The processor 105 of the interactive vehicle safety system 100 maycontrol and process various actions for the interactive vehicle safetysystem 100 as will be described further below. The processor 105 may bea general-purpose processor, a digital signal processor (DSP), anapplication-specific integrated circuit (ASIC), a field programmablegate array (FPGA) or other programmable logic device, discrete gate ortransistor logic, discrete hardware components, or any combinationthereof designed to perform the functions described herein. Ageneral-purpose processor may be a microprocessor, or any conventionalprocessor, controller, microcontroller, or state machine. A processormay also be implemented as a combination of computing devices, e.g., acombination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. The one or more implementationsdescribed throughout this disclosure may utilize logical blocks,modules, and circuits that may be implemented or performed with theprocessor 105.

The processor 105 may be used to implement various aspects and featuresdescribed herein. As such, the processor 105 may be configured toexecute multiple calculations, in parallel or serial and may executecoordinate transformations, curve smoothing, noise filtering, outlierremoval, amplification, and summation processes, and the like. Theprocessor 105 may include a processing unit and system memory to storeand execute software instructions. The processor 105 may be incommunication with and/or connected to the interactive vehicle safetysystem 100 that may provide a central analysis and display.

FIGS. 7A1-7D2 illustrate the interactive vehicle safety system 100 withtrajectory and dead-reckoning analysis and a heads-up display 270A alongthe full windshield 18. FIGS. 8A-8D illustrate the interactive vehiclesafety system 100 with trajectory and dead-reckoning analysis and aheads-up display 270B along a portion of the windshield 18, and thelower left-hand portion of the windshield 18. In another embodiment ofthis invention, the interactive vehicle safety system 100 may includetrajectory and dead reckoning analysis. For example, dead reckoning isthe process of calculating a vehicle's current position and/or futureposition by using a previously determined position, or current position,and by using estimation of speed and course over elapsed time. Theinteractive vehicle safety system 100 may calculate a vehicle's futureposition by using a current position and by using estimation of speed,course, and other inputs over elapsed time. For collision avoidance, thereaction time may be approximately 2 seconds with a possible stoppingdistance of 3-5 seconds. The interactive vehicle safety system 100 mayinclude real-time image processing for a vehicle to detect and track inreal-time object movement outside the vehicle. The interactive vehiclesafety system 100 may also highlight stationary or moving objects aroundthe vehicle that are predicted to be a danger utilizing trajectoryanalysis and dead reckoning of the vehicle and the stationary or movingobject. The trajectory analysis and dead reckoning analysis may utilizespeed, direction, acceleration, turn radius, etc. from the vehicle andspeed, direction, acceleration, etc. from the moving object. Theprediction of the route and path of the vehicle and the location of themoving object may be calculated by the interactive vehicle safety system100 by various methods, such as algorithms using speed, direction, turnradius, acceleration, GPS, vehicle telematic data and sensors, cameras,external sensors, mapping information, etc. Additionally, theinteractive vehicle safety system 100 may include a predictive algorithmutilizing data and information to watch what pedestrians are doing,looking at the phone or wearing headphones or tracking the pedestrian'sacceleration to determine if they are walking, about to run, or about tostop. The interactive vehicle safety system 100 may also utilize machinelearning with the image processing to provide better predictivealgorithms for the trajectory analysis.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may include a dead-reckoning heads-up display 270 to includeultrasound sensors or other sensors to the front bumper, rear bumper, orother locations around the vehicle. As illustrated in FIGS. 7A1-7D2, theinteractive vehicle safety system 100 provides a dead reckoning display270A with information and warnings fully across the windshield and areal time picture across the windshield that can be displayed in variousdifferent colors (i.e. gray for dead reckoning and another color forreal time). As illustrated in FIGS. 7A1-7D2, the heads-up display 270Amay include both the vehicle/obstacles or text 274, such as “WARNING” asshown in the heads-up display 270A.

First, the interactive vehicle safety system 100 removes the A-pillarand other structural pillars to provide a clear, complete, open view tothe operator of the vehicle 10. As an external, moving object's (such asa pedestrian 22) direction and speed is detected, the interactivevehicle safety system 100 displays the object and the object speed anddirection both on the clear metal screen 260 but also across thewindshield 18 by way of the heads-up-display 270A on the windshield 18or dash 16. By including ultrasonic sensors on the front bumper, therear bumper, or other locations around the vehicle, the interactivevehicle safety system 100 may then utilize real-time trajectory andobject movement analysis and bring the dead reckoning into the real timespace on the heads-up-display. The interactive vehicle safety system 100may include a dead reckoning strip of LED pictures across the front ofthe dashboard 16 of the vehicle 10 and reflecting onto the windshield18.

FIGS. 7A1 and 7A2 illustrate the interactive vehicle safety system 100with trajectory and dead-reckoning analysis showing a second vehicle 20turning left in front of a main vehicle 10. Specifically, FIG. 7A1 showsthe second vehicle 20 preparing to turn left in front of the mainvehicle 10. FIG. 7A2 shows the heads-up-display 270B with the secondvehicle 20A turning in front of the main vehicle 10 and “WARNING” 274 byutilizing the trajectory and dead-reckoning analysis of the interactivevehicle safety system 100.

FIGS. 7B1 and 7B2 illustrate the interactive vehicle safety system 100with trajectory and dead-reckoning analysis showing a main vehicle 10turning left in front of a second vehicle 20. Specifically, FIG. 7B1shows the main vehicle 10 preparing to turn left in front of the secondvehicle 20. FIG. 7B2 shows the heads-up-display 270B with the mainvehicle 10A turning in front of the second vehicle 20 and “WARNING” 274by utilizing the trajectory and dead-reckoning analysis of theinteractive vehicle safety system 100.

FIG. 7C illustrates the interactive vehicle safety system 100 withtrajectory and dead-reckoning analysis showing a pedestrian 22 walkingin a crosswalk in front of a main vehicle 10. As illustrated in FIG. 7C,the pedestrian 22 is behind and blocked by the A-pillar 14. The operatoris able to view the pedestrian 22 in the crosswalk because of thedisplay 260 from the pillar obstruction elimination system 250. Theheads-up-display 270B may display “WARNING” 274 by utilizing thetrajectory and dead-reckoning analysis of the interactive vehicle safetysystem 100.

FIGS. 7D1 and 7D2 illustrate the interactive vehicle safety system 100with trajectory and dead-reckoning analysis showing a main vehicle 10turning left in front of a pedestrian 20 in a crosswalk. Specifically,FIG. 7D1 shows the main vehicle 10 preparing to turn left in thedirection of the pedestrian 22 in the crosswalk. FIG. 7D2 shows theheads-up-display 270B with the main vehicle 10A turning towards thepedestrian 22 in the crosswalk along with “WARNING” 274 by utilizing thetrajectory and dead-reckoning analysis of the interactive vehicle safetysystem 100.

FIGS. 8A-8D illustrate the interactive vehicle safety system 100 withtrajectory and dead-reckoning analysis and a heads-up display 270B alonga portion of the windshield 18, and the lower left-hand portion of thewindshield 18. As illustrated in FIGS. 8A-8D, the heads-up display 270B(along with the heads-up display 270A) may include both thevehicle/obstacles 272 or text 274, such as “WARNING” as shown in theheads-up display 270B. Specifically, FIG. 8A illustrates the interactivevehicle safety system 100 with the heads-up-display (HUD) 270B in thelower left-hand portion of the windshield 18 showing a second vehicle 20turning left in front of a main vehicle 10. FIG. 8B illustrates theinteractive vehicle safety system 100 with a heads-up-display (HUD) 270Bin the lower left-hand portion of the windshield 18 showing a mainvehicle 10 turning left in front of a second vehicle 20. FIG. 8Cillustrates the interactive vehicle safety system 100 with aheads-up-display (HUD) 270B in the lower left-hand portion of thewindshield 18 showing a pedestrian 22 walking in a crosswalk in front ofa main vehicle 10. FIG. 8D illustrates the interactive vehicle safetysystem 100 with a heads-up-display (HUD) 270B in the lower left-handportion of the windshield 18 showing a main vehicle 10 turning left infront of a pedestrian 22 in a crosswalk.

The highlighting of the moving object in danger may be highlighted invarious stages. For example, the moving object may be highlighted yellowif the object is potentially in the path of accident or collision withthe vehicle. In addition, the moving object may be highlighted red ifthe object is moving and imminently in the path of accident or collisionwith the vehicle. Highlighting may be in the form of different colors,blinking colors, circles around the object, blinking circles, etc,without departing from this invention.

The interactive vehicle safety system 100 may utilize the heads-updisplay 270 as described and illustrated previously or the interactivevehicle safety system 100 may utilize a dashboard display 280. FIG. 9illustrates an exemplary dashboard display 280 located on the dashboard16 of the vehicle 10. The dashboard display 280 may include both thevehicle/obstacles 282 or text 284, such as “WARNING” as shown in thedashboard display 280. The interactive vehicle safety system 100 mayutilize any one of or any combination of the heads-up display 270A, theheads-up display 270B, or the dashboard display 280 without departingfrom this invention to display what is happening outside the vehicle orwhat might happen outside the vehicle.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may provide a full vision of what is happening outside of thevehicle 10. For example, the interactive vehicle safety system 100 willprovide and improve an operator's peripheral visual awareness to providesituational awareness to the operator. The interactive vehicle safetysystem 100 may utilize one or more of the following systems andinformation to provide and improve the operator's peripheral visualawareness. For example, the interactive vehicle safety system 100 mayprovide an augmented reality system. The interactive vehicle safetysystem 100 may also provide depth cameras or other system that providedepth imagery allowing the ability to change the perspective of theoperator's view. For example, the augmented reality system and/or thedepth cameras may provide a display with the driver's perspective “infront” of the vehicle or with the driver's perspective “on top” of thevehicle. Depth cameras may also not return colors and may return agrey-scale image to help determine depth and therefore the distance of apedestrian, object, or other danger object in the path of the vehicle.

In another embodiment of the invention, the operator may also utilizeglasses, contacts, or a circular plastic cover that drops over the faceand eyes to provide an augmented reality vision of the full vision ofwhat is happening outside of the vehicle. The cover may drop over theface and eyes of the operator when required or when initiated by theinteractive vehicle safety system 100. The augmented reality system mayinitiate the movement of the cover based on a projected trajectory of acollision and/or accident. The augmented reality system may initiate themovement of the cover upon movement of the vehicle.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may utilize various sensors to provide the operator andsystem with additional information and situational awareness. Thevarious sensors may include one or more of the following sensors:density, vibration, audio, humidity, air pressure, color, syntheticsensors, etc. The various sensors may be located on the front bumper,rear bumper, or other locations around the vehicle. The one or moresensors may include trajectory sensors to help determine and providedata and analysis of the trajectory and dead reckoning of the vehicleand any moving objects external to the vehicle. The one or more sensorsmay also include synthetic sensors wherein one sensor senses an actionand the other sensors as part of the synthetic sensor confirms thataction.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may utilize stereoscopy with the plurality of cameras. Theplurality of cameras may be located throughout the exterior of thevehicle, such as in front, sides, back, top, or bottom of the vehicle.The stereoscopy may utilize two or more cameras to accurately determinedepth, location, and trajectory of moving objects or pedestriansexternal to the vehicle.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may determine and track the location of the operator's headto change the view for the operator's view based on the operator's headlocation when moving, rotating, or at different heights. Tracking theoperator's head location will help maintain aspect ratio location of theoperator's vision of the display and external from the interactivevehicle safety system 100.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may integrate data from various other information sources.The plurality of information sources may be one or more informationsources on the Internet of Things (IoT), such as from camera informationfrom intersections, buildings, autonomous vehicles, or other camerasources, sensors, or measuring devices throughout the area.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may include image analysis with cameras that can detectionwhat a pedestrian is doing. For example, image analysis may detectearbuds and/or headphones or a pedestrian talking on a cell phone. Theinteractive vehicle safety system 100 may utilize this image analysisinformation to potentially take a different action.

In another embodiment of this invention illustrated in FIG. 10, theinteractive vehicle safety system 100 may utilize hologram technologywith motion parallax. As illustrated in FIG. 10, the hologram system mayprovide a hologram 30 on the pillar 14 of the pedestrian 22 locatedbehind the pillar 14. For example, as the operator moves eye direction,the foreground may shift faster than the background and create astereoscopic view. The hologram technology may utilize a laser, one ormore beam splitter mirrors (such as two mirrors), one or more lens (suchas three lens), and a holograph film located on the pillar 14 or otherlocation within the vehicle 10. By using a laser light, the light may bein the same direction and same wavelength to provide coherent beams withall light waves in phase to project the hologram 30 or 3D version ofwhat is happening outside and blocked by the driver's vision.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may provide audible and/or visual warnings to pedestriansand/or other danger objects. For example, the interactive vehicle safetysystem 100 may provide audible, visual, and other warnings (such assounds, visual, and/or motion—such as a vibrating seat) to both thepedestrians and the vehicle operator when collision, an accident, ordanger is possible. The interactive vehicle safety system 100 mayutilize a transducer or other systems to send directed audible warningsto the danger pedestrian, such as 3D sounds.

In another embodiment of this invention as illustrated in FIG. 11, theinteractive vehicle safety system 100 may provide an audio “spotlight”290 for the operator. In this embodiment, an aimed audio device 292located on the vehicle 10 casts the audio “spotlight” 290 to a surfaceredirecting the source/sound and pointed specifically at that spot, suchas the sound coming from that spot. This allows the interactive vehiclesafety system 100 to provide an audible warning 294 to the operatorcoming from the location of a potential accident or collision locationbased on the trajectory analysis, sensors, and cameras with theinteractive vehicle safety system 100.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may provide other warnings to the operator and/orpedestrians. For example, the interactive vehicle safety system 100 mayprovide a vibrating seat or haptic warning to the operator when apotential collision or accident is detected. In another example, theinteractive vehicle safety system 100 may provide external, automatic,audible warnings outside of the vehicle, such as, “Warning—Vehicleapproaching” or “Warning—Vehicle turning.” The interactive vehiclesafety system 100 may provide internal, automatic, audible warningsinside of the vehicle to the operator, such as, “Warning—Pedestrian incrosswalk.” The interactive vehicle safety system 100 may also includean audio system with “white” noise or nuisance noise, ambient noisesensor to change external sounds, or GPS geofencing.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may include a highlighting feature 240 on the display or onthe external area 40 when the vehicle is making a turn. For example, aswas described above, the interactive vehicle safety system 100 maydisplay the path of the turn on the display for the operator based onthe trajectory analysis, sensors, and cameras with the interactivevehicle safety system 100. Additionally, as illustrated in FIG. 12, theinteractive vehicle safety system 100 may externally, outside of thevehicle 12, light up the external path 240 on the pavement or externalarea 40 of where the vehicle 10 is going, such as the trajectory ofgoing straight or a turn during the turn. For example, the interactivevehicle safety system 100 may laser the path 240 and the direction ofthe vehicle 10 on the street or pavement 40—painting a turn or goingstraight with high-intensity LED lights 242 or similar lightingelements. To assist with optically displaying the path 240 externally onthe street or pavement 40, the interactive vehicle safety system 100 mayalso provide infrared and heat-detection sensors to help “block-out” theroad to display the external projected path of the vehicle on the road.

The interactive vehicle safety system 100 may include optical sensors ona steering wheel for determining a turning radius of a turn andproviding predictive modeling on the turning path of the vehicle.Additionally, the interactive vehicle safety system 100 may includesensors on the wheels of the vehicle, such as an electromagnetic arm orpitmen arm for determining a predictive analysis of the turning radiusof the vehicle. Additionally, back-up cameras or front cameras mayprovide additional turning trajectory analysis and path projection ofthe vehicle.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may provide visual information and visual warnings. Thevisual warning may include a strobe or laser directed specific to a“danger” pedestrian to alert the pedestrian of an oncoming, turningvehicle. Additionally, the interactive vehicle safety system 100 mayinclude a heads-up-display (HUD) on the windshield or other locationbased on the various cameras and sensors associated with the interactivevehicle safety system 100. The heads-up-display may include any of theinformation as described to include information about various movingobjects and stationary objects from the trajectory analysis determinedas potential collision or accident. The heads-up-display may include adistance or how many feet or inches an object is away from the vehicleand/or collision. The heads-up-display may also include a time tocollision based on the trajectory analysis of the vehicle and the movingobject.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may include a processor and database for recording andstoring information and images from turns and actions based on “danger”present or imminent—critical moments. The critical moments may bedetermined by geofencing, accelerometer analysis, and impact detection.The interactive vehicle safety system 100 may create a 3D model from thecameras and sensors to recreate an accident. This information can bevery helpful in a number of situations as providing evidence of theactual actions that occurred during these critical moments.

In another embodiment of this invention, the interactive vehicle safetysystem 100 may include a number of automatic actions in response to animminent or present danger situation. For example, the interactivevehicle safety system 100 may automatically stop acceleration of thevehicle at a “yellow” condition for a collision or accident. In anotherexample, the interactive vehicle safety system 100 may automaticallyapply the brakes on the vehicle at a “red” condition for a collision oraccident. The “yellow” and “red” conditions may be set by parameterswithin the interactive vehicle safety system 100. The interactivevehicle safety system 100 may require the vehicle to maintain within thespeed limit, utilizing GPS-location services or even a camera thatrecognizes the speed limit through image analysis. In another example,the interactive vehicle safety system 100 may utilize other automaticactions, such as: pre-emptive braking, changing steering direction, hornhonking, flashing lights, or vibration in the seats to help assist withmaintain vehicle safety.

It is to be understood that the invention is not limited in itsapplication to the details of construction and the arrangement of thecomponents set forth herein. The invention is capable of otherembodiments and of being practiced or being carried out in various ways.Variations and modifications of the foregoing are within the scope ofthe present invention. It should be understood that the inventiondisclosed and defined herein extends to all alternative combinations oftwo or more of the individual features mentioned or evident from thetext and/or drawings. All of these different combinations constitutevarious alternative aspects of the present invention. The embodimentsdescribed herein explain the best modes known for practicing theinvention and will enable others skilled in the art to utilize theinvention.

While the preferred embodiments of the invention have been shown anddescribed, it will be apparent to those skilled in the art that changesand modifications may be made therein without departing from the spiritof the invention, the scope of which is defined by this description.

1-26. (canceled)
 27. An interactive safety system for a vehiclecomprising: one or more image capturing devices located on the vehicleand configured to transmit one or more images from the vehicle to aprocessor; one or more object detecting sensors located on the vehiclethat detect location, speed, and direction data of one or more objectsexternal to the vehicle, the one or more object detecting sensorsconfigured to transmit the location, speed, and direction data from ofthe one or more objects to the processor; a display system connected tothe processor, the display system comprising: a pillar display locatedon an A-pillar of the vehicle configured to portray an image blocked byan obstruction of the A-pillar from the one or more image capturingdevices; the processor and a memory unit storing computer-executableinstructions, which when executed by the processor, cause the processorto: receive images from the one or more image capturing devices; receivethe location, speed, and direction data from the one or more objectdetecting sensors; process in real-time the images and the location,speed, and direction data from of the one or more objects external tothe vehicle; track in real-time the one or more objects external to thevehicle using the images and the location, speed, and direction data;predict a future location and a route of the one or more objectsexternal to the vehicle using a predictive algorithm and a trajectoryanalysis of the one or more objects external to the vehicle; determinein real-time a danger object from the one or more objects external tothe vehicle, wherein the danger object is a potential collision based onthe predictive algorithm and the trajectory analysis; display the imageblocked by the obstruction of the A-pillar on the pillar display; andprovide an audible warning to an operator coming from a location of thedanger object, wherein the audible warning is an audio spotlight from anaimed audio device connected to the processor that casts a sound to asurface redirecting the sound to come from the location of the dangerobject relative to the vehicle.
 28. The interactive safety system ofclaim 27, wherein the computer-executable instructions further cause theprocessor to: provide a visual warning to the danger object, wherein thevisual warning is provided external to the vehicle.
 29. The interactivesafety system of claim 27, wherein the computer-executable instructionsfurther cause the processor to: light a path of the vehicle withhigh-intensity lights based on the predictive algorithm and trajectoryanalysis, wherein the path is externally outside the vehicle on apavement of a street.
 30. The interactive safety system of claim 27, thedisplay system further including a heads-up display on a windshield ofthe vehicle to display the one or more objects with the location, speed,and direction data and the danger object with the future location andthe route on a heads-up display.
 31. The interactive safety system ofclaim 27, wherein the computer-executable instructions further cause theprocessor to: provide an audible warning to the danger object, whereinthe audible warning is provided external to the vehicle.
 32. Theinteractive safety system of claim 31, wherein the audible warning is a3D sound directed to the danger object.
 33. The interactive safetysystem of claim 27, wherein the one or more object detecting sensors areultrasonic sensors.
 34. The interactive safety system of claim 27,wherein the one or more object detecting sensors are LIDAR radarsensors.
 35. The interactive safety system of claim 27, wherein the oneor more object detecting sensors are photoelectric sensors.
 36. Theinteractive safety system of claim 27, the display system furtherincluding a rear-view display located in the vehicle and configured toportray an image from the one or more image capturing devices that wouldbe seen in a rear-view mirror, wherein the computer-executableinstructions further cause the processor to: display the image thatwould be seen in the rear-view mirror on the rear-view display.
 37. Theinteractive safety system of claim 27, further comprising one or moretelematics devices configured to transmit vehicle telematics data fromthe vehicle to the processor, wherein the computer-executableinstructions further cause the processor to: receive the telematics datafrom the telematics device; and process in real-time the telematicsdata, wherein the one or more objects external to the vehicle aretracked further using the telematics data.
 38. The interactive safetysystem of claim 27, wherein the computer-executable instructions furthercause the processor to: highlight the danger object on the displaysystem.
 39. The interactive safety system of claim 38, wherein thedanger object is highlighted yellow if the danger object is potentiallyin a path of an accident or a collision.
 40. The interactive safetysystem of claim 38, wherein the danger object is highlighted red if thedanger object is imminently in a path of an accident or a collision withthe vehicle.
 41. The interactive safety system of claim 38, whereinhighlighting the danger object includes one or more of the following:blinking colors on the danger object, circling the danger object, orblinking circles around the danger object.